The Performance Potential of Trace-based Dynamic Optimization

Dynamic optimization can apply powerful optimizations to hot execution paths that span traditional boundaries such as branches and calls, including calls to dynamic libraries. Because of this opportunity, it has gained academic and industrial attention. In this paper, we examine dynamic optimization in an overhead-free environment to ascertain its optimization potential. Specifically, we explore an ideal implementation with zero overhead and ask: what is the performance potential of trace-based dynamic optimization? Using a methodology consisting of a trace selector, optimizer, and trace-driven simulation framework, we provide insights on this potential. We begin by demonstrating the potential of dynamically-applied classical optimizations as a function of trace length. Dynamic optimizers identify hot execution paths better than static mechanisms and, as a side-effect, are able to create longer optimization regions. In this analysis, we quantify contributing performance factors and identify optimization hindrances and their cost. We perform cross-ISA analysis of SPARC and x86, demonstrate significant differences between the two, and explain the basis for the differences. We also examine performance, trace cache size, and instruction stream coverage effects for ideal and real-world trace selectors and estimate the remaining optimization potential available.